Transport Layer Security (TLS) - Grokking the API Design Interview

Motivation#

We have established that APIs are significant for businesses and their smooth functioning. However, to ensure their successful operation, we need a layer of security to protect data in various communication channels. The transmission of requests and subsequent responses between a client and server must be secured. Otherwise, attackers can intercept these transmissions, which can subsequently lead to information disclosure and tampering.

Assume the following simplistic view of a client-server communication that is being performed using unsecured public Wi-Fi. Without the application of any relevant security protocols, the messages we send and the API requests and responses generated are left vulnerable to any attacker over the network.

Referring to the diagram above, we need to be able to answer the following questions to guarantee a secure transmission:

How does the client know that the server it’s talking to is exactly the one the client intended?
How do we achieve confidentiality/secrecy of the messages?
How does the client know about the integrity of the message it has received?
How do we verify the authenticity of messages received from the server?

So, we need to utilize a framework/model that ensures the desired security properties during transmission to prevent attacks.

Transport layer security (TLS)#

Transport layer security (TLS) is a cryptographic protocol that permits safe transmission between the client and API provider. TLS ensures message authentication, encryption, and data integrity.

In a client-server communication, the server usually requests authentication from clients using sensitive credentials, like a username and password, an API key, and tokens (we’ll explore these techniques in the coming lessons). However, using TLS, the client is able to authenticate the server prior to client authentication. TLS achieves this with the help of digital certificates that a client receives directly from the server.

Right now, it may appear that encryption is TLS’ most essential feature. However, the identification and the subsequent prevention of attacks are just as important. Encryption isn’t of much use if we aren’t sure of who we’re talking to, as an attacker may be altering the transmission. Therefore, before we dive into the details of the functionality of TLS, let’s jog our memory of:

Symmetric and asymmetric cryptography
Public key infrastructure

Functionality#

As we observed above, asymmetric encryption is too slow to be used for data encryption, whereas symmetric encryption has no built-in mechanism for secure key exchange between two parties. Therefore, TLS combines both symmetric and asymmetric encryption in establishing the handshake protocol between the client and server.

Note: Asymmetric encryption will be utilized for secure exchange of symmetric keys, which in turn encrypts the data. With this strategy, we can now guarantee confidentiality of messages exchanged between clients and servers.

From the discussion above, we are able to answer the first and second questions we raised in the "Motivation" section earlier. Now, let's expand on how TLS deals with the other three questions by examining how it functions:

The client sends an initial message to the server. The message contains its TLS/SSL version, along with the client random (a string of randomized bytes) and the cryptographic algorithm it supports.
The server then responds with a cryptographic algorithm chosen from the list it obtains from the client, its session ID, the server’s random, and its certificate, which contains an asymmetric public key for encryption purposes (public key exchange). The hash for the entire communication up until the server hello message is encrypted with the server’s private key to produce a digital signature. Because no other entity is in possession of the private key, only the server can produce the signature. Therefore, the client can be sure of the message’s authenticity through these digital signatures.
The client verifies the validity of the certificate provided through the relevant certification authority, which confirms the authenticity of the web server. This is how the server authenticates itself to the client.
Client authentication into the server is optional because it’s usually handled outside the TLS protocol. Here, the digital signature is how the client verifies the server’s message authenticity. The server and client authenticate each other’s identity in steps 3 and 4.

Note: The client and server random (both are strings) are used to calculate the digital signature. A digital signature proves the authenticity of the server to the client. Also, it validates that the data between the client and server hasn’t been tampered. If an attacker attempts to alter any information, the client will not be able to validate the digital signature, and the handshake will have to be restarted.

The client generates another string called the premaster secret, which is then encrypted through the server’s public key and sent to the server. The server decrypts the premaster secret by using the corresponding private key. Now both parties can generate the shared symmetric key by using the server random, client random, and the premaster secret, arriving at the same result. The generated symmetric key derives aspects from both the client and the server, making it unique, by incorporating their randoms (the client random and server random). This is the process through which the symmetric key is securely obtained on both sides.
The client sends a “finished” message that indicates the client’s end of the handshake is complete. This is the first message that is encrypted with the shared secret key.
The server responds with its own “finished” message that indicates that the server’s end of the handshake is complete. The server and client can now freely exchange any application data that will be encrypted through the shared symmetric key.

Note: The generated symmetric key is only valid for that particular session and will have to be generated again if the session ends. For instance, if the client closes the application through which it established the connection, it would have to go through this entire process again the next time it opens the application. This is done to prevent any security vulnerabilities that attackers may exploit.

TLS handles message integrity by making use of message authentication codes (MACs). A MAC ensures that the message hasn't been altered during transmission, usually with secure hash functions (such as SHA 256). Hashed MACs (HMACs) differ in that they specifically utilize secret keys, which are present at both the client and the server, to hash the data. The process is illustrated below:

The hash generated through the use of a secret key is appended to the original message. This concatenated entity is then encrypted by the shared symmetric key. This message is sent to the client, which decrypts this message, calculates its own hash of the message (using the secret key), and compares the two hashes. If they aren't identical, the message has been tampered with and is discarded. The client may then request the message to be sent again. This is the process through which message integrity is upheld in TLS.

Points to Ponder

Question 3

If data is encrypted between the client and the server, why do we need HMACs?

Hide Answer

HMACs are needed to confirm the integrity of the message and whether the data has been corrupted during transmission. It functions differently from TLS in that it doesn’t implement any encryption for confidentiality but exclusively checks if the messages have been altered. Attackers don’t always alter messages. For instance, packets may be dropped in an encrypted session. For this use case, HMACs are required.

3 of 3

Summary of TLS#

TLS answers all the questions we presented at the beginning of this lesson:

How does the client know that the server it’s talking to is exactly the one the client intended? Through the use of digital certificates, the client and server are identified.
How do we achieve confidentiality/secrecy of the messages? Confidentiality is achieved with the encryption protocols implemented because even if an attacker intercepts a transmission, they won't be able to comprehend or decrypt the ciphertext.
How does the client know about the integrity of the message it has received? Integrity is achieved when the receiving party checks the MAC to see if the message has been altered.
How do we verify the authenticity of messages received from the server? The client can be sure of the server's message authenticity by evaluating digital signatures.

TLS has become commonplace not only in securing communications in APIs but in web applications in general.

HTTP vs. HTTPS#

Before TLS, HTTP protocol was the standard for web communication due to its lightweight nature. However, it forwards data over the Internet in plaintext, which causes it to be insecure and susceptible to attackers.

When we communicate through HTTPS, we are provided with another layer of security, which is provided through SSL/TLS. This not only transmits encrypted data instead of plaintext, but it also uses TLS for supplementary defenses. The usage of these protocols causes it to be heavier than HTTP, which leads to poorer performance.

Let’s say we send a GET request to an entity. If the HTTP protocol was used and the attacker intercepts the transmission, they'll obtain the request similar to the following:

The unsecured nature of HTTP has caused it to be overtaken by HTTPS because virtually all communication on the Internet is encrypted in some way, usually through TLS.

Conclusion#

The introduction of TLS in applications for security induces little performance degradation. However, with its advantages, it's a no-brainer to implement TLS into our API's communication with web servers/clients. Newer versions of TLS support techniques that speed up the handshake process, such as TLS False Start, which allows the client and the server to begin communication before the handshake is completed. The issue is that TLS False Start isn't compatible with all TLS clients and servers due to poorly implemented servers. Therefore, False Start can be used to increase performance, but only if certain conditions are met.

These developments associated with TLS have allowed the computational costs and load times to be negligible, which is critical in regard to API design.

Note: Nowadays, it’s recommended to use TLS 1.3 because it has better performance and is more secure than its predecessors. It does this by reducing round trip times and dropping support for older vulnerable cryptographic algorithms.

Protocol	Advantages	Disadvantages
HTTP	Lightweight, better for performance	Sends in plaintext, not secure
HTTPS	Encrypted, secure	Worse performance than HTTP